1. Technical Field
This invention relates to neural computing systems and methods and, more particularly, to a reconfigurable system for emulating multiple neural network architectures.
2. Discussion
Biologically inspired artificial neural networks have opened up new possibilities to apply computation to areas that were previously thought to be the exclusive domain of human intelligence. Neural networks learn and remember in ways that resemble human processes. Areas which show the greatest promise for neural networks, such as pattern classification tasks like speech and image recognition, are areas where conventional computers have had the greatest difficulty.
In general, artificial neural networks are systems composed of many nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural nets. The computational elements, or nodes, are connected via variable weights that are typically adapted during use to improve performance. Thus, in solving a problem, neural net models can explore many competing hypothesis simultaneously using massively parallel nets composed of many computational elements connected by links with variable weights. In contrast, with conventional von Neumann computers, an algorithm must first be developed manually, and a program of instructions written and executed sequentially. In some applications, this has proved extremely difficult. This makes conventional computers unsuitable for many real-time problems.
The implementation of neural network systems, however, has lagged somewhat behind their theoretical potential due to the difficulties in building neural network hardware. This is primarily because of the large numbers of neurons and weighted connections required. The emulation of even of the simplest biological nervous systems would require neurons and connections numbering in the millions. Due to the difficulties in building such highly interconnected processors, the currently available neural network hardware systems have not approached this level of complexity. Another disadvantage of hardware systems is that they typically are custom designed and built to implement one particular neural network architecture and are not easily, if at all, reconfigurable to implement different architectures.
To overcome these limitations in current neural network hardware, a number of software simulations of neural networks have been developed. Software systems have the advantage of being easier to reconfigure than hardware systems. However, since software simulations are performed on conventional sequential computers, they do not take advantage of the inherent parallelism of neural network architectures. Consequently, they are relatively slow. One frequently used measurement of the speed of a neural network processor is the number of interconnections it can perform per second. For example, the fastest software simulations available can perform up to about 18 million interconnects per second. Such speeds, however, currently require expensive super computers to achieve. Even so, 18 million interconnects per second is still too slow to perform many classes of pattern classification tasks in real time. These include radar target classifications, sonar target classification, automatic speaker identification, automatic speech recognition and electro-cardiogram analysis, etc.
Thus, it would be desirable to provide a system for emulating a neural network that is relatively easy to reconfigure for the simulation of a variety of neural network architectures. Further, it would be desirable to provide a neural network system which employs the inherent parallelism in neural networks to improve the speed of computation so that pattern classification problems can be performed in real-time.
A further consideration is that neural computation generally involves two steps. The first step is the learning procedure in which interconnection weights are repeatedly adjusted to improve the performance of the system. Once learning is complete, in the second step the neural network operates on a given problem to produce a desired solution. After training, the interconnection weights are known and problems can be solved with a system having fixed weights. By separating the learning procedure from the operating procedure, a fixed weight processor system would be much simpler to construct and faster to operate than a trainable system. Thus, to achieve real-time neural computation, it would be desirable to have a system dedicated to the performance of neural computation on known, fixed weights.